Skip to content

feat(gateway): a not_ready 503 says the lease's place in line - #327

Merged
T-Crypt merged 1 commit into
nextfrom
T-Crypt/queue-position-headers
Nov 13, 2026
Merged

T-Crypt merged 1 commit into
nextfrom
T-Crypt/queue-position-headers

Conversation

@T-Crypt

@T-Crypt T-Crypt commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Milestone 3 of the Berth Next plan: make waits honest.

Problem

When a gateway start timed out (not_ready), a non-streaming client got a 503 with Retry-After: 30 and nothing
else, even while its lease stood in line for the card. The client could not tell "the engine is broken" apart from
"you are second in line behind a training".

Change

When the start times out, the gateway reads berthd's status once and finds the engine's waiting lease on its card.
If one is in line, the 503 now carries:

  • X-Berth-Queue-Position: N, where 1 is next in line.
  • X-Berth-Start-Estimate: N; lower-bound, the broker's estimate_start_s rounded up. It is missing when nothing
    ahead was measured.
  • Retry-After, the larger of the estimate and 30 s.

The error message adds "its lease is number N in line for the card".

The estimate is a lower bound because the broker counts only the measured starts ahead in the queue and never the
holder's time left (internal/broker/queue.go). Behind a pinned training it is far too low. The header says so
rather than presenting it as a promise. A Retry-After that adds the holder's time needs the hold lengths that the
plan's decision log will record.

Streaming requests already got their headers when the SSE preamble went out, so they keep the same error as an SSE
event and get no new headers.

Proof

  • TestNotReadyCarriesTheQueue runs a real test berthd with a pinned hold on the card, so the gateway's spawned
    berth run truly waits. Without the change it fails with queue "", Retry-After "30" and a bare not_ready
    message.
  • TestQueuedStartHeaders serves a fake status with another engine ahead in line. It checks the rounding (44.2 s
    gives 45), the 30 s floor, a missing estimate, and that only the engine's own lease counts.
  • proof.sh passes in full: build, gofmt, vet, go test ./..., the Windows build and vet, and golangci-lint 2.14.0.

docs/gateway.md describes the headers under the failure rules.

A non-streaming request whose start timed out got a bare 503 with
Retry-After 30, even while its lease stood in line for the card. The
gateway now reads the lease's place in line from berthd when the start
times out, and the 503 carries X-Berth-Queue-Position and, when the
broker has measured the starts ahead, X-Berth-Start-Estimate: N;
lower-bound. Retry-After is the larger of the estimate and 30 s.

The broker's estimate counts only the measured starts ahead in the
queue and not the holder's time left, so the header marks it as a
lower bound instead of passing it off as a promise.
@T-Crypt
T-Crypt merged commit 284f233 into next Nov 13, 2026
4 checks passed
@T-Crypt
T-Crypt deleted the T-Crypt/queue-position-headers branch November 13, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant